Solving differential problems using full order models (FOMs), like the finite element method, incurs prohibitively computational costs in real-time simulations and multi-query routines. Reduced order modeling aims at replacing FOMs with reduced order models (ROMs), that exhibit significantly reduced complexity while retaining the ability to capture the essential physical characteristics of the system. In this respect, latent dynamics models (LDMs) represent a novel mathematical framework in which the latent state is constrained to evolve according to an (unknown) ODE. A time-continuous setting is employed to derive error and stability estimates for the LDM approximation of the FOM solution. The impact of using an explicit Runge-Kutta scheme in a time-discrete setting is then analyzed, resulting in the $\Delta$LDM formulation. Additionally, the learnable setting, $\Delta$LDM_{\theta}, is explored, where deep neural networks approximate the discrete LDM components, ensuring a bounded approximation error with respect to the high-fidelity solution. Moreover, the framework demonstrates the capability to achieve a time-continuous approximation of the FOM solution in a multi-query context, thus being able to compute the LDM approximation at any given time instance while retaining a prescribed level of accuracy.